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Claims 

We claim: 



1 LA method for providing a virtual reality environment, comprising: 

2 acquiring concurrently, with a plurality of cameras, a plurality of 

3 sequences of input images of a 3D object, each camera having a different 

4 pose; 

5 reducing the plurality of sequences of images to a differential stream 

6 of 3D operators and associated operands; 

7 maintaining a 3D model of point samples representing the 3D object 

8 from the differential stream, in which each point sample of the 3D model has 

9 3D coordinates and intensity information; 

10 rendering the 3D model as a sequence of output image of the 3D 



1 1 object from an arbitrary point of view while acquiring and reducing the 

12 plurality of sequences of images and maintaining the 3D model in real-time. 

1 2. The method of claim 1, in which the acquiring and reducing are 

2 performed at a first node, and the rendering and maintaining are performed 

3 at a second node, and further comprising: 

4 transmitting the differential stream from the first node to the second 

5 node by a network. 

1 3. The method of claim 1, in which the object is moving with respect to the 

2 plurality of cameras. 
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1 4. The method of claim 1, in which the reducing further comprises: 

2 segmenting the object from a background portion in a scene; and 

3 discarding the background portion. 

1 5. The method of claim 1, in which the reducing further comprises: 

2 selecting, at any one time, a set of active cameras from the plurality of 

3 cameras. 

1 6. The method of claim 1, in which the differential stream of 3D operators 

2 and associated operands reflect changes in the plurality of sequences of 

3 images. 

1 7. The method of claim 1, in which the operators include insert, delete, and 

2 update operators. 

1 8. The method of claim 1, in which the associated operand includes a 3D 

2 position and color as attributes of the corresponding point sample. 

1 9. The method of claim 1, in which the point samples are rendered with 

2 point splatting. 

1 lO.The method of claim 1, in which the point samples are maintained on a 

2 per camera basis. 

1 11. The method of claim 1, in which the rendering combines the sequence of 

2 output images with a virtual scene. 
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1 12. The method of claim 1, further comprising: 

2 estimating a local density for each point sample. 

1 13. The method of claim 1, in which the point samples are rendered as 

2 polygons. 

1 14. The method of claim 1 , further comprising: 

2 sending a silhouette image corresponding to a contour of the 3D 

3 object in the differential stream for each reduced image. 

1 15. The method of claim 1, in which the differential stream is compressed. 

1 16. The method of claim 1, in which the associated operand includes a 

2 normal of the corresponding point sample. 

1 17. The method of claim 1, in which the associated operand includes 

2 reflectance properties of the corresponding point sample. 

1 18. The method of claim 1, in which pixels of each image are classified as 

2 either foreground or background pixels, and in which only foreground pixels 

3 are reduced to the differential stream. 

1 19. The method of claim 1, in which attributes are assigned to each point 

2 samples, and the attributes are altered while rendering. 

1 20. The method of claim 19, in which the point attributes are organized in a 

2 vertex array that is transferred to a graphics memory during the rendering. 
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